Skip to content

fix(relay): recover interrupted outbox rebuilds and retain local fact identities - #29

Draft
McHersheys wants to merge 7 commits into
pullboard-dev:mainfrom
McHersheys:fix/receipt-preserving-rebuild
Draft

McHersheys wants to merge 7 commits into
pullboard-dev:mainfrom
McHersheys:fix/receipt-preserving-rebuild

Conversation

@McHersheys

@McHersheys McHersheys commented Nov 13, 2026 •

Copy link
Copy Markdown

Related: #14, #30, and the interrupted-rebuild follow-up on #15. DRAFT, unmerged. Source publication is not acceptance, live recovery, or a permanent legacy-ID compatibility fix.

New explicit service-first recovery candidate

Head dcc92fa adds an operator-selected loss-tolerant route for a principal who explicitly chooses to abandon unpublished local messages rather than reconstruct their history. Ordinary behavior is not silently changed. This is separate from lossless legacy recovery, which remains unresolved below.

The operator API requires the exact inspected queue digest and attributable loss selection. It reconstructs only a same-board authenticated contiguous ordered prefix using the native engine, checks all existing receipt/refusal values, stages a protected recovery plan and existing recovery hold, retains the original sealed queue outside active sending, and uses the native atomic restore with source-digest comparison. It preserves board identity and credentials and sends no old move. Same-selection completion readback does not reset later work. Native event cursor and relay position remain distinct. Original uncertain effects are not declared absent or successful.

Qualification of this exact source: 37/37 focused real-SQLite/sealed-queue controls PASS, zero skips, Node24.21.0. Twelve new controls cover held/archived/restored persisted cuts, changed input/plan/board refusals, receipt preservation, no POST, and useful new native item/claim after recovery. The persisted-cut controls are same-process reopens, not literal power-loss evidence. An initial fixture omitted its required brief/check and failed four continuation assertions; that genuine failure was retained before correcting the fixture.

A remaining-only offline rehearsal against an actual retained queue and authenticated ordered bridge also passed: no unsettled reapplication or transport mutation, original ordered receipts exact, source native digest already identical to the reconstructed ordered baseline, queue retired byte-for-byte, credentials retained, native cursor correct. Producer joined and disposable copy removed. This does not bind a current live tail or establish native coordinator continuation. No private captures or generated outputs are in this PR.

New independent review is pending. Runtime preconditions remain real: stop the affected scheduler normally; establish same-board relay-lock/outbox-guard exclusion including stale local-first callers and publisher behavior; bind exact current inputs; independently accept the chosen artifact; and reconcile the same coordinator's obsolete scheduling cursor/requests after the intentional history loss. The source API is not a claim that these operational gates are already satisfied. No serving restart or shared-runtime replacement is part of this change.

Retained predecessor corrections and contrary evidence

01f68f7 received PASS_SCOPED independent review for preserving the legacy identity transport veto across refused reapplication/persisted retries and using canonical implied seeds for legal73/80-character move IDs. Its25 focused controls passed with genuine old-code RED evidence. That scoped result does not establish full recovery. Prior local-rebuild improvements recover a covering authenticated prefix, validate captured identity inputs before mutation, preserve particular lease deadlines and refuse demonstrated initial positional shifts.

The prior full-history rehearsal FAILED fidelity:239 nested clock fields and7201 unsettled timestamps remained drift, despite correct settled receipts and many non-clock fields. Later producer-induced numeric-reference shifts and all original application clocks remain unresolved. Existing authoritative replay still demonstrates correction loss where random local fact IDs were never encoded in the sealed original move. Local overrides or future-only IDs cannot supply absent authenticated inputs to other replicas.

Lossless recovery therefore still requires the owner-selected compatibility requested in #30. The new explicit abandonment option does not solve, alias, rewrite or conceal that defect. No applicable full gate, merge/release acceptance, live restore or useful coordinator continuation is claimed. Exact-source independent review and runtime-bound adoption remain owed.

@McHersheys

Copy link
Copy Markdown
Author

The first offline rehearsal against the retained real backlog reconstructed every item identity, preserved all existing ordered receipts, and retained all 5,081 unacknowledged move payloads unchanged. Its transport refused before the first unsent move; nothing was sent or installed. This is not yet full recovery acceptance: some recent review states differed, and that fixture omitted 460 genuine native proof-receipt files that are all present at the origin. The same candidate is now being checked with those exact files and per-refusal diagnostics.

Independent review of the forward-only reconstruction and receipt-preservation checks would be useful. This remains draft; no production adoption or full-gate claim.

@McHersheys

Copy link
Copy Markdown
Author

The real-backlog rehearsal with all genuine proof receipts still FAILED full fidelity. It recovered item identities and preserved the already-ordered receipts, but that is not sufficient to install or drain. No live board changes or sends were made.

I isolated and fixed one additional client defect in be4ed01: pending-history reconstitution renewed old claims using the current wall clock. Two originally valid claims separated by lease expiry became ONE_CLAIM when reconstructed together. The new regression is RED before the fix and PASS after it; all 10 focused reconstruction tests pass. The control also proves authoritative replay still uses the actual relay timestamp and records ONE_CLAIM when appropriate. Missing original times refuse before undo/restore. This remains draft, without full-gate or independent-review acceptance.

There is a separate unresolved reference-identity defect, now reproduced with two ordinary queued facts and a real SQLite/sealed-outbox fixture: queue appendFact A, then a genuine correction B with supersedes=A's returned id. Both succeed locally. After interrupted reconstruction, executableArgs replaces A's local random fact id with A's sealed move id; B still references the original id and disappears from the rebuilt projection with NO_FACT. That missing event then causes observedThrough activity receipts to exceed the rebuilt prefix. The real backlog also contains answerDecision targets now pointing to different non-decision shouts. Those must not be waved away as acceptable recovery.

The next owning source/review question is receipt-preserving binding of original fact/shout/event references across provisional reconstruction and actual ordering, including already-created queues. A prospective origin-ID fix alone cannot recover that history. I have stopped another expensive unchanged whole-backlog run: the small counterexample fails in about 1.5 seconds, while the lease/source controls still pass. Please review this seam alongside the reconstruction candidate; do not merge this as complete recovery or recommend another drain yet. The original move identities/order/ciphertext and all failed/unknown histories remain preserved.

@McHersheys

Copy link
Copy Markdown
Author

Related: #30. The permanent fact-ID regression is now tracked separately. The current published head be4ed01 does not fix it: the retained two-fact correction test still fails because reconstruction changes the original successful local fact identity. I am extending this same repair line to preserve existing queued identities and qualify dependent references; no identity-fix, independent-acceptance, or live-restoration claim yet.

@McHersheys McHersheys changed the title fix(relay): recover an interrupted outbox rebuild from retained ordered history fix(relay): recover interrupted outbox rebuilds and retain local fact identities Nov 13, 2026
@McHersheys

Copy link
Copy Markdown
Author

Related #30. The draft now includes 3595c01: the explicit opt-in local recovery API holds the existing relay lock and reentrant outbox guard continuously, binds retained link inputs across a held retry, and returns the original recovered native-event cursor/digest on completion readback. Recovery holds no longer suggest an unrelated --skip undefined operation.

A discriminating predecessor test reproduced acceptance of changed pending link state after a held cut. The changed helper passes 13 focused recovery cases. Three additional public-API tests exercise real file-backed SQLite and link persistence, including held/archived retries; they use explicitly synthetic authenticated GET responses, not a live relay. The installed-package qualification also checks the actual native lock implementation and preserves unrelated installed cache fixes. Original failed checks remain retained outside source Git.

The changed integration is undergoing independent review. This is not a claim of live restoration, general lossless legacy-ID compatibility, a released upgrade or native acceptance. Service-first local message abandonment is an explicitly authorized operator option; it neither settles uncertain effects nor permits replay of already-acknowledged moves. Genuine old-caller/checkpoint-publisher quiescence and owner-controlled adoption remain required.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant